Reconstruction, Registration, and Modeling of Deformable Object Shapes

Biological or biomedical objects, such as expressive human faces and growing brain tumors, and dynamic scenes, such as cars running on the roads, generally vary their shapes as linear combinations of a number of shape bases. With the expeditious development of computer and imaging technologies, the problems of reconstruction, registration, and modeling of such deformable shapes from image measurements has shown enormous importance for applications such as biomedical image interpretation, human computer interaction, and robot navigation. Since the image measurements are generated by coupling two factors: non-rigid deformations and rigid similarity transformations between the shapes and the measurement systems, the essence of the three problems is to factorize the shape measurements and compute the deformable shapes (reconstruction), the rigid transformations (registration), and the shape bases (modeling). This thesis presents novel factorization algorithms respectively for reconstruction, registration, and modeling of deformable shapes. We also demonstrate an application of the proposed algorithms for modeling and fitting of expressive human faces. Firstly, we study the problem of reconstructing 3D deformable shapes from 2D images, assuming the weak-perspective camera model and non-degenerate cases. The previous methods usually enforce the constraints on the orthonormality of the rigid rotation transformations (rotation constraints) alone to solve this problem. The thesis quantitatively shows that enforcing the rotation constraints alone is inherently insufficient and yields ambiguous and invalid solutions. We point out that the ambiguity stems from the non-uniqueness of the shape bases, and introduce the basis constraints that resolves the ambiguity by implicitly specifying a unique set of bases. The thesis proves that, enforcing the basis constraints, together with the rotation constraints, results in a linear closed-form solution to factorizing the image measurements and simultaneously reconstructing the rigid transformations, the deformable shapes, and the underlying shape bases. We also develop methods for reconstruction of degenerate or planar deformations that are described by rank-2 or rank-1 shape bases. Specifically, we quantitatively show that enforcing the basis and rotation constraints still achieves a linear closed-form solution when all the degenerate bases are of rank 1, but results in an ambiguous solution space when rank-2 bases are involved. We then develop an alternating linear optimization method for reconstruction of rank-2 deformations. Secondly, we develop a 2-step factorization algorithm for reconstruction of 3D deformable shapes from uncalibrated images under the full perspective camera model. In the first step, we recover the projective depths using the sub-space constraints embedded in the deformable shape measurements. In the second step, we first scale the image measurements by the reconstructed projective depths. The scaled measurements are then factorized by an extension of the linear closed-form algorithm for